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Carnegie  Mellon 

Software  Engineering  Institute  CMM 

Brief  History  -  CMMI 

1992  -  Software  CMM  created 

1994  -  Systems  Engineering  CMM  created 

1998  -  CMMI  Product  Suite  initiated 

2001  -  CMMI-SE/SW  VI  .0  released 

2002  -  CMMI-SE/SW/IPPD/SS  VI  .1  Product  Suite 

released 

2003  -  10,000  people  trained  in  “Intro  to  CMMI;”  150+ 

SCAMPI  benchmark  appraisals  in  at  least  12 
countries;  CMMlweb  site  “hits”  exceed  1  M/month 
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Carnegie  Mellon 

Software  Engineering  Institute  CMMI 

CMMI  Today 

Stable  Version  1.1  CMMI  Product  Suite  was  released 
January  2002. 

Errata  sheets  cover  known  errors  and  changes  with  book 
publication. 

FAQs  are  generated  to  cover  broader  issues. 

Yahoo  has  CMMI  Process  Improvement  and  Lead 
Appraiser  Group  sites. 

CMMI  web  pages  hits  have  surpassed  1  M/month. 

Change  Request  announcement  addressed  90  day  review 
period  through  Dec  12. 
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SW-CMM  vl.1  vs.  CMMI  Process  Areas 


f  > 

LEVEL  5 

Defect  Prevention  - 

— ►  Causal  Analysis  and  Resolution 

^  Ornanivatinnal  lnnn\/atinn  &  Plonlri\/mont 

OPTIMIZING 

Process  Change  Management  — 

w  1  MCI  1  1 IZ-C4  LI  Ul  1 Q 1  IIMIvJVClUvJII  CX  L»  CUIUy  llld  1  L 

LEVEL  4 
MANAGED 

Quantitative  Process  Mgmt 
Software  Qualitv  Mamt 

Organizational  Process  Performance 
Quantitative  Project  Management 

Organization  Process  Focus 
Organization  Process  Definition 

Organization  Process  Focus 

Organization  Process  Definition 

Training  Program 
Integrated  Software  Mgmt 

Software  Product  Engr 


LEVEL  3 
DEFINED 


Intergroup  Coordination 
Peer  Reviews 


J  Regi  lirements  Management 


Organizational  Training 
Integrated  Project  Management 

Risk  Management 
Requirements  Development 
^  Technical  Solution 
Product  Integration 
Verification 
Validation 

Decision  Analysis  and  Resolution 

Rpqi  lirements  Management 


LEVEL  2 
REPEATABLE 


Software  Project  Planning 
Software  Project  Tracking  &  Oversight 
Software  Subcontract  Mgmt 
Software  Quality  Assurance 
Software  Configuration  Mgmt 


Project  Planning 
Project  Monitoring  and  Control 
Supplier  Agreement  Management 
Product  &  Process  Quality  Assurance 
Configuration  Management 
Measurement  and  Analysis 
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CMMI 


CMMI  Improvements  over  the  CMM 


Emphasis  on  measurable  improvements  to  achieve 
business  objectives. 

Process  areas  have  been  added  to  place  more 
emphasis  on  some  important  practices: 

•  Risk  Management 

•  Measurement  and  Analysis 

•  Engineering  Process  Areas 

•  Decision  Analysis 
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Carnegie  Mellon 

Software  Engineering  Institute 

Adoption — What  else  is 
happening  now? 


CMMf 

y 


Publication  of  SEI  Series  Book  with 
Addison-Wesley:  others  include: 

•  CMMI  Distilled:  Second  Edition 

•  Systematic  Process  Improvement  Using 
ISO  9001:2000  and  CMMI 

•  Balancing  Agility  and  Discipline 

Annual  NDIA/SEI  CMMI  User  Workshop 

•  Denver  Hyatt  Technical  Center 

•  Nov  17-20 
400+  attendees 

Mappings  taken  on  by  IEEE 


i  CMMI 

i  Guidelines  for  Process 
I  Integration  and  Product 
*  Improvement 


Mary  Beth  Chrissis 
Mike  Konrad 
Sandy  Shrum 
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Carnegie  Mellon 

Software  Engineering  Institute  CMM 

How  about  SEI  Publications? 

Technical  notes  and  special  reports: 

•  Interpretive  Guidance  Project:  Preliminary  Report 

•  CMMI  and  Product  Line  Practices 

•  CMMI  and  Earned  Value  Management 

•  Interpreting  CMMI  for  Operational  Organizations 

•  Interpreting  CMMI  for  COTS  Based  Systems 

•  Interpreting  CMMI  for  Service  Organizations 

•  Providing  Safety  and  Security  Assurance  (in  progress) 

•  Interpreting  CMMI  for  Acquisition  (in  progress) 
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Carnegie  Mellon 

Software  Engineering  Institute 

CMMI  Transition  Status 

As  of  12/31/03 

Training 

Introduction  to  CMMI  -  10103  trained 
Intermediate  CMMI  -  777  trained 
Introduction  to  CMMI  Instructors  -  219  trained 
SCAMPI  Lead  Appraisers  -  379  trained 


Authorized 

Introduction  to  CMMI  VI. 1  Instructors  - 176 
SCAMPI  VI  .1  Lead  Appraisers  -  267 
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Carnegie  Mellon 

Software  Engineering  Institute 

Number  of  CMMI  Students 
Trained  (Cumulative) 


CMMI 
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4000 
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0 
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CMMI 


Number  of  Lead  Appraisers 

450 
400 
350 
300 
250 
200 
150 
100 
50 
0 


(Cumulative) 


1994  1995  1996  1997  1998  1999  2000  2001  2002  YTD 

2003 


Authorized 


□  Lead  Evaluators 
(SCE) 

■  Lead  Assessors 
(CBA IPI) 

□  Lead  Appraisers 
(SCAMPI) 
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Carnegie  Mellon 

Software  Engineering  Institute 


Intro  to  the  CMM  and  CMMI  Attendees 

(Cumulative) 


CMMI 
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2003 


□  CMM  Intro 

□  CMMI  Intro 

■  CMMI  Intermediate 
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CMJIff 


Number  of  Assessments  Reported  to  the  SEI  by  Year 
30  September  2003 


1987  1988  1989  1990  1991  1992  1993  1994  1995  1996  1997  1998  1999  2000  2001  2002  2003 

~D~SPA  □  SEI  CBA  IPI  □  SCAMPI  vX 


©2003  by  Carnegie  Mellon  University 


page  12 


Carnegie  Mellon 
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CMJIff 


CMMI®  Results  Study  Framework 


Costs 


Benefits  /  Savings  /  Improvements 


Process 
Capability 
or  Maturity 

Rework 

Effort 

Quality 

Productivity 

Schedule  / 
cycle  time 

Predictability 

Enhanced 

functionality 

_ i 

Product  cost 

“ilities” 

Customer 

satisfaction 

Employee 

morale 

Process 

compliance 
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Carnegie  Mellon 

Software  Engineering  Institute 

Boeing,  Australia 


CMMI 


Making  transition  to  CMMI  from  SW-CMM  and  EIA  731; 
early  CMMI  pilot  in  Australia 


RESULTS  on  One  Project 

•  33%  decrease  in  the  average  cost  to  fix  a  defect 

•  Turnaround  time  for  releases  cut  in  half 


Product  cost 


Schedule  / 

60%  reduction  in  work  from  Pre-Test  and  Post-Test  Audits; 
passed  with  few  outstanding  actions 


•  Increased  focus  on  product  quality 

•  Increased  focus  on  eliminating  defects 

•  Developers  seeking  improvement  opportunities 


Quality 


In  Processes  is  there  a  Pay-Off?  Terry  Stevenson,  Boeing  Australia,  Software  Engineering 
Australia  2003  conference. 
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^  Carnegie  Mellon 

Software  Engineering  Institute 

Lockheed  Martin  M&DS 

SW  CMM  ML2  (1993)  to  ML  3  (1996)  to  CMMI  ML5 
(2002) 

Results 

•  Award  Fees  during  2002  are  45%  percent  of 
unrealized  award  fees  at  ML2 


1996-2002 

•  Increased  software  productivity  by  30% 

•  16%  reduction  in  Dollars/KLOC 

•  Decreased  defect  find  and  fix  costs  by  15% 


Customer 

satisfaction 


Productivity 
Product  cost 


Quality 


Internal  data  shared  through  Collaboration;  August  2003. 
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Software  Engineering  Institute 


CMMI 


General  Motors  Corporation 

CMMI  focus  2001 

Goal  is  Integration  of  Supplier  Work  &  GM  Project  Execution 


Results: 

•  Improved  schedule  -  projects  met  milestones  and 
were  fewer  days  late 


Schedule  / 
cycle  time 


Camping  on  a  Seesaw:  GM’s  IS&S  Process  Improvement  Approach.  Hoffman,  Moore  & 
Schatz,  SEPG  2003. 
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CMMI 


Aggregated  Appraisal  Results 


Plow*  Mgmgentt  nt 


Guals 


G  oa  I  Bali  da  die  n 


Generic  Goal 


Majority  d  organisation*  appraised  achieved  Goal  Satidadiun 


$p*o!fie  Goal 


Majority  of  organisation*  appraised  did  not«hi*u<  ■Goal 


Satisfaction 


;Not  Applicable 


Pnoctss  Ar?a*  of  immediate  Commit 


||  | 

69% 

100% 

BB% 

BB% 

1  66%1 

I  DPF  1  CPC  1  IPM 

PP 

PMC  i  $AU 

RI$K 

m 

pie 

TS 

PI 

V£R 

VAL 

PPGAI  CM  |  OAR  1 

Results  from  18  Defence  Community*  appraisals  conducted  over 
the  period  Mid  2000  -  Present 

•  includes  Defence  Industry  and  Department  of  Defence  appraisal 
results 


(C)  Cup/idglri  nDiiritrijiri  wMitHi  ui Jklta'to'&Hfl  -  SspismbsT  2003 
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Carnegie  Mellon 

Software  Engineering  Institute  CMM 

The  Road  Ahead.... 

Formal  Review  period  ends  mid-December 

CMMI  Team  will  review  CRs  to  determine  possible 
Change  Packages  for  a  VI  .2  of  model  and/or  method 

CCB  will  determine  which  CPs,  if  any,  are  needed 
(stability  goal  remains) 

Improvement  Packages  would  be  an  FY  05  effort,  with 
piloting 

VI  .2  would  be  ~FY  06 
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Carnegie  Mellon 

Software  Engineering  Institute 

CMMI  Staged  and  Six  Sigma 


CMMI 
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Carnegie  Mellon 

Software  Engineering  Institute  CMI 

Six  Sigma  and  CMMI  Continuous 

Achieve  high  capability  in  PAs  that  build  Six  Sigma  skills. 
•  MA,  QPM,  CAR,  OPP 

Use  capability  to  help  prioritize  remaining  PAs 


Remaining  PAs  ordered  by  business  factors,  improvement  opportunity,  etc.  which  are 
better  understood  using  foundational  capabilities.  CMMI  Staged  groupings  and  DMAIC 
vs.  DMADV  are  also  factors  that  may  drive  the  remaining  order.  [Vickroy  03] 
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Carnegie  Mellon 

Software  Engineering  Institute 


LOCKHEED  M  A  1^^ 


LMC  M&DS  Process  Standard 
Roadmap 


Six  Sigma  links: 

[s-p  03]  Level  2  Measurement  &  Analysis  PA,  Level  4/5  PAs 
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CMMI 


ix  Sigma  Approach  at  Northro, 
Grumman 


Level  2  Self  Assessma*  &  Monitoring 

EXAMPLE  OF  TOO.  USAGE 

8 

“ 

ft 

0™.ToM 

Linked  with  Business 
Planning  and 
Oversight 
•Business  planning 
•Project  selection 


Quantitatively  Driven 
•  Six  Sigma  improvements  are  integrated  with  Quality 


quantified 


Enabled  by 
Infrastructure. 

Training 
Tools 

•Awareness 
Database  Tied  to  Employee 
Performance 
•Goals,  awards 
•J  ob  and  career  paths 


Program 
■Integrated  Training, 
Awareness,  &  Policies 
■Integrated  CMMI  &  Six 
Sigma  projects 
■Integrated  tracking  and 
reporting  via  DB,  PRA,  etc. 


Name: 

nickname: 

Job  Title: 

— 

'location:  ^ 

Six  S»qrn<i  Status: 

Employee  ID: 

Minitab.  Ink 


Engaged  with 
External 
Customers 
•Visibility 
•Participation 
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Carnegie  Mellon 

Software  Engineering  Institute 


CMMf 


Northrop  Grumman’s  Six  Sigma 
Implementation 

Started  implementing  Six  Sigma  in 
2001 

Trained  over  3000  Green  Belts  (80 
hours),  and  over  200  Black  Belts 
(160  hours) 

Completed  several  hundred  projects 
covering  all  functional  areas 
•  Customer  involvement  and  award 
fee  citations 

About  half  of  the  projects  are 
improving  an  engineering  process 
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Carnegie  Mellon 
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CMMI 


3  Keys  to  Competitive  Leverage  at 
Northrop  Grumman 


SixSigma  is  a 
business  strategy  to 
deliver  value  and 
develop  a  sustainable 
competitive 
advantage 


KM  provides  a 
strategy 

to  utilize  data  and 
transform  it  into 
knowledge  to 
enable 
informed  and 
decisive 
management 
leadership 
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Process  Maturity  Profile 

CMMI®  vl  .1 

SCAMPISMv1.1  Appraisal  Results 


Mike  Phillips 

CMMI  Program  Manager 
Softv\j0re  Engineering  Institute 
Carnegie  Mellon  University 


The  Software  Engineering  Institute  is  a  federally  funded  research  and  development  center  sponsored  by  the  U.S.  Department  of  Defense  and  operated  by  Carnegie  Mellon®  University 
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CMMf 

_ ^ 


Introduction 
Current  Status 
Summary 

Terms  used  in  this  Briefing 

How  to  Report  your  Appraisal  Results  to  the  SEI 
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Carnegie  Mellon 

Software  Engineering  Institute 

Introduction:  Purpose 

Characterize  the  adoption  of  the  CMMI 

Describe  results  from  Standard  CMMI  Appraisal  Method 
for  Process  Improvement  (SCAMPI  vl.1)  Class  A 
appraisals  using  Capability  Maturity  Model  Integration 
(CMMI)  vl.1  * 

Encourage  continued  reporting  of  results 


*  Organizations  previously  appraised  against  CMMI  vl.O  and  who  have  not  reappraised 
against  vl.1  are  not  included  in  this  report 


Please  visit:  http://www.sei.cmu.edu/sema/profile_about.html  for  additional  information 
or  questions  you  may  have  about  this  briefing  before  contacting  the  SEI  directly 
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Carnegie  Mellon 

Software  Engineering  Institute  CM  IV 

Current  Status 

SCAMPI  vl.1  appraisals  conducted  since  April  2002 
release  ar|d  reported  to  the  SEI  by 

•  130  appraisals 

•  123  organizations 

•  68  participating  companies 

•  gji  reappraised  organizations 

•  44%  projects 

•  offshore  organizations 

Please  refer  to:  Terms  Used  in  this  Report  on  page  21 
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Reporting  Organizational  Types 
1 


Commercial/ln-house 


Contractor  for 
Military/Government 


Military/Government 

Agency 


49.6% 


0%  10% 


20%  30%  40% 

%  of  Organizations 


50%  60% 


Based  on  123  organizations 
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Reporting  Organizational  Types  - 


0%  5%  10%  15%  20%  25%  30%  35%  40%  45% 

%  of  Organizations 


Based  on  123  organizations 
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Carnegie  Mellon 
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Types  of  Organizations 


CMMI 


Based  on  Primary  Standard  Industrial  Classification  (SIC) 

J  Retail  Trade  ....  .  '  * 

Code 


Retail  Trade 
1.6% 


Wholesale  Trade 
1 .6% 


Engineering  &  Management 
Services 
27.0% 


Business  Services 
27.0% 


Finance,  Insurance  and  Real 
Estate 
4.8% 


Public  Administration  (Including 
Defense) 

1 1 .1  % 


Industrial  Machinery  And 
Equipment 
3.2% 


Transportation  Equipment 
4.8% 


Electronic  &  Other  Electric 
Equipment 
7.9% 


Instruments  And  Related 
Products 
7.9% 


Health  Services 
1.6% 


Services,  Nec 
1 .6% 


Based  on  63  organizations  reporting  SIC  code.  For  more  information  visit:  http://www.osha.gov/oshstats/sicser.html 
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Carnegie  Mellon 

Software  Engineering  Institute 


Organization  Size 


CMMf 


Based  on  the  total  number  of  employees  within  the  area  of  the  organization 

that  was  appraised  2000+ 

4.9% 


25  or  fewer 


201  to  300 


6.6% 


Based  on  122  organizations  reporting  size  data 
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Use  of  Model  Representations  in 
Appraisals 


w 

TO 

SO 

'ro 

Q. 

a 

< 


a) 

-Q 

E 

3 


120 


100 


80 


60 


40 


20 


Staged 


Continuous 


Based  on  136  appraisals 
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Disciplines  Selected  for 
Appraisals 


ro 

(A 

'fB 

a 

a 

< 


a) 

n 

E 

3 


SW 

=  Software  Engineering 

SE 

=  System  Engineering 

SS 

=  Supplier  Sourcing 

IPPD 

=  Integrated  Product  and 

Process 

Development 

SW  +  SE 


SE  +  IPPD  + 
SS 


SE  +  SS 


Based  on  136  appraisals  reporting  coverage 
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Carnegie  Mellon 

Software  Engineering  Institute 


CMMf 


Summary  Organizational 


Initial  Managed  Defined  Quantitatively  Optimizing 

Managed 


Based  on  most  recent  appraisal  of  87  organizations  reporting  a  maturity  level  rating 


©2003  by  Carnegie  Mellon  University 


page  35 


%  of  Organizations 


Carnegie  Mellon 

Software  Engineering  Institute 


Maturity  Profile  by 
Organizational  Type 


^  . 

CMMf 


Initial  Managed  Defined  Quantitatively  Optimizing 

Managed 


□  Commercial/ln-house  II  Contractor  for  Military/Government  □  Military/Government  Agency 

87 

Based  on  most  recent  appraisal  of  organizations  reporting  organization  type  and  a  maturity  level  rating 
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Carnegie  Mellon 

Software  Engineering  Institute 

Countries  Where  Appraisals 
Have  Been  Performed  and 
Reported  to  the  SEI 


Australia  Canada  China  Colombia  France  India 

Japan  Korea,  Republic  of  Russia  Switzerland  Taiwan  United  Kingdom 

United  States 
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Carnegie  Mellon 

Software  Engineering  Institute 


USA  and  Offshore  Summary 
Organizational  Maturity  Profiles 


CMMI 


100% 

90% 

80% 

70% 

60% 

50% 

40% 

30% 


27.8% 


54.5% 


29.6% 


Initial  Managed  Defined  Quantitatively 

Managed 


□  USA  □  Offshore 

Based  on  33  U.S.  organizations  and  54  offshore  organizations  reporting  their  maturity  level  rating 


Optimizing 
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Carnegie  Mellon 

Software  Engineering  Institute 


CMMf 


Process  Area  Satisfaction  -  Maturity 


REQM  PP  PMC  SAM  MA  PPQA  CM 


119  121 


121 


93 


118 


119  120 


Based  on  the  number  of  appraisals  listed  above  that  rated  the  process  area 
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Process  Area  Satisfaction  -  Maturity 


RD  TS  PI  Ver  Val  OPF  OPD  OT  IPM  *RSKM  IT  ISM  DAR  OEI 


89  88 


84  92  85  91 


90  88 


83  89  13  15  82  11 


Based  on  the  number  of  appraisals  listed  above  that  rated  the  process  area 


11  of  the  83  appraisals  rating  IPM  also  examined  the  IPPD  goals.  10 
of  these  11  appraisals  satisfied  IPM  with  the  IPPD  goals. 
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Process  Area  Satisfaction  -  Maturity 
Levels  4&5 


Based  on  the  number  of  appraisals  listed  above  that  rated  the  process  area 
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Defined 
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Process  Area  Profiles  - 1 
Organizations  Appraised  at  Maturity 
4evel  1 


%  of  Appraisals 


Based  on  12  appraisals  reporting  a  maturity  level  rating 


*  None  of  the  14  appraisals  selected  the  IPPD  discipline 
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Optimizing 

CAR 

Quantitatively 

OID 

Managed 

QPM 
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Process  Area  Profiles  -  2 
Organizations  Appraised  at 
jritv  Level  2 


CMMf 

y 


Defined 


100% 


% 


Based  on  22  appraisals  reporting  a  maturity  level  rating 


of  Appraisals 

*  1  of  the  22  appraisals  rating  IPM  also  examined  the  IPPD  goals. 
That  appraisal  satisfied  IPM. 
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CMMI 

_ ^ 


Relatively  even  reporting  from  the  Commercial  and  Contractor 
communities,  however  Commercial  organizations  are  primarily 
outside  of  the  U.S.  and  Government  Contractors  are  primarily 
located  in  the  U.S. 

Of  U.S.  organizations,  the  services  and  manufacturing  industries 
reported  most  appraisals. 

Compared  to  the  early  reports  of  the  SW-CMM  maturity  profile,  the 
early  data  reflects  a  relatively  more  mature  CMMI  profile. 

Additional  information  and  charts  will  be  added  to  this  briefing  as 
more  appraisals  are  reported  and  therefore  more  data  is  available  to 
support  these  breakdowns. 
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Company  -  Parent  of  the  appraised  entity 

A  company  can  be  a  commercial  or  non-commercial 
firm,  for-profit  or  not  for-profit  business,  a  research 
and  development  unit,  a  higher  education  unit,  a 
government  agency,  or  branch  of  service,  etc. 


Organization  -  a.k.a.  Appraised  entity 

The  organization  unit  to  which  the  appraisal  results 
apply.  An  appraised  entity  can  be  the  entire  company, 
a  selected  business  unit,  units  supporting  a  particular 
product  line  or  service,  etc. 

Offshore  -  Appraised  entity  whose  geographic  location  is  not 

within  the  United  States.  The  parent  of  the  appraised 
entity  may  or  may  not  be  based  within  the  United 
States. 
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Report  your  Appraisal  Results 
to  the  SEI 


CMMI 


The  briefing  is  only  possible  due  to  the  cooperation  of 
organizations  and  individuals  sending  in  their  appraisal 
results  to  the  SEI 

In  order  to  provide  this  information  and  service  in  the  future, 
it  will  depend  on  this  continued  cooperation 

Please  visit: 

http://www.sei.cmu.edu/sema/report.html 

for  forms,  information,  and  instructions  on  how  to  report 
appraisals  to  the  SEI 
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Contact  Information 

Please  visit: 

http://www.sei.cmu.edu/sema/profile_about.html 

and  review  the  information  provided  before  contacting: 

SEI  Customer  Relations  (412)  268-5800 
SEI  FAX  number  (412)  268-5758 

Internet  Address 

customer-relations@sei.cmu.edu 

Mailing  Address 
Customer  Relations 
Software  Engineering  Institute 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213-3890 
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